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Executive Summary 

BUILDING ASSETS REDUCING RISKS 

Final Report 

This report presents the final results of the U.S. Department of Education Investing in 
Innovation (i3) grant, The Building Assets-Reducing Risks Model: Replication and Expansion of 
an Effective Strategy to Turn Around Low-Achieving Schools. 

The Building Assets Reducing Risks (BARR) Model BARR is a comprehensive 
model that addresses the challenges that are part of the 9 th grade transition year through the 
implementation of eight different school-wide and individual strategies. It combines teachers’ 
real-time analysis of student data, student asset building, and intensive teacher collaboration 
to prevent course failure. It reaches all students and teachers, and uses SEL skills to help 
increase academic performance. It develops positive student-teacher relationships and 
integrates student supports into a school's existing model for addressing non-academic 
barriers to learning. 

BARR has been implemented at St Louis Park High School, MN since school year 1998 
- 1999. It was initially funded through a Minnesota Department of Human Services State 
Incentive Grant. In 2009, SAMHSA’s National Registry of Evidence-based Programs and 
Practices (NREPP) listed the BARR program in its registry, and has recently been recognized in 
the CASEL Guide for Effective Social and Emotional Learning Programs. In 2010, BARR 
received an Investing in Innovation (i3) Development grant from the US Department of 
Education to replicate and expand BARR to other high schools and conduct a randomized 
controlled trial to test its effectiveness. This report details the final results of the i3 Development 
grant. 

A large suburban high school in southern California participated in a within-school 
Randomized Controlled Trial (RCT) of the BARR Model. A total of 555 9 th grade students were 
randomly assigned to BARR and non-BARR conditions. At the end of the RCT year, BARR 
students had earned significantly more core course credits, higher grade point averages, and 
had a lower course failure rate than non-BARR students. BARR students also earned 
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significantly higher standardized test scores on the Northwest Education Association’s (NWEA) 
Measures of Academic Progress (MAP) scores in mathematics and reading than did non-BARR 
students after one year of implementation; specifically an average of two years of growth in 
mathematics, compared to one year of decline in the non-BARR condition. In addition, students 
in the BARR condition who were identified as having lower academic proficiency earned 
significantly more core course credits and achieved higher NWEA scores in mathematics and 
reading than did their counterparts in the non-BARR condition. Students identified as more 
academically proficient earned significantly higher NWEA scores in mathematics than did their 
counterparts in the non-BARR condition. 

In the second and third years of the grant, BARR was implemented in the entire 9 th 
grade, and the core course failure rate continued to decline. In addition, the achievement gap 
between Hispanic and non-Hispanic students closed by year two of implementation and 
remained closed in year three. Implementation fidelity of the BARR model was achieved in year 
one, and continued to improve over the second and third years of the study. 

BARR was also implemented in two rural high schools in Maine. One high school 
implemented BARR with fidelity and saw decreases in core course failure rate, increases in 
grade point averages, and increases in standardized test scores in reading, language, and 
mathematics during the three years of BARR implementation. The other high school did not 
achieve fidelity in the quality of implementation, and there were no significant decreases in 
failure rate nor consistent increases in standardized test scores over the three years. 

BARR teachers reported improved relationships with students, increased ability to 
perceive student strengths, use of data to improve student performance, better communication 
with administration, less isolation, and better problem solving of problematic student issues. 
Results were seen for both new and veteran teachers. New teachers felt supported and veteran 
teachers felt rejuvenated. Teachers in the schools where implementation fidelity was not 
achieved rated aspects of the program significantly lower than did teachers in the other two 
schools. 
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II - Background 


The Importance of 9 th Grade 


Ninth grade is a pivotal year for students. Numerous studies document that academic 
performance in 9 th grade often sets the student's trajectory throughout the high school years, 
as well as the probability of graduation (e.g., Benner, 2011; Neild, 2009; Weiss & Baker- 
Smith, 2010). For example, students are 3-5 times more likely to fail a class in 9th grade than 
students in any other grade (Southern Regional Educational Board, 2002). Almost one 
quarter of students in the top quartile of their eighth grade class were off track by the end of 
9 th grade (Allensworth & Easton, 2005). Twice as many high-achieving low-income students 
failed to graduate on time compared to high-achieving high-income peers (Wyner et al, 2007). 
Overall, 70-80% of students who fail in 9 th grade will not graduate from high school (Wyner et 
al, 2007). 

The transition to 9 th grade includes developmental, academic, and structural 
challenges. Neild (2009) proposes several possible explanations for why students get off 
track in the 9 th grade: 

1) Developmental and life course changes where parental influence wanes, and 
children have more autonomy, reduced parental supervision and support while peer influence 
increases. These can lead to increased risk taking and declining academic performance. 

2) Transition to a new school that involves breaking the social bonds that students 
had formed with their teachers and peers from the middle grades. Students must negotiate 
new social relationships and adapt to the practices and routines of the new school. 

3) Inadequate preparation for high school where students who struggled 
academically or who were inadequately challenged before high school fall even further 
behind. Students with poor math and reading skills are overwhelmed by academic demands 
of high school and get discouraged that they will ever complete it. 

4) High school organization and climate in which the traditional social organization 
of high school encourages teachers to focus on the subject matter and not the students. 
Students have different teachers for each subject, and teachers have little or no opportunities 
to learn how students are doing in other classes (Benner, 2011; Neild, 2009). 

Given these challenges and the critical nature of the 9 th grade year, several programs 
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have emerged to help students make a successful transition to high school. These programs 
include Project Transition (Neild, 2009), Talent Development (Kemple, Herlihy, & Smith, 
2005), Diplomas Now (Balfanz, Bridgeland, Fox, DePaoli, Ingram, & Maushard, 2014; Corrin 
& Sepanik, 2014), On-track Indicator developed by UChicago CCSR ( Allensworth & Easton, 
2005; Roderick, Kelley-Kemple, Johnson, & Beechum, 2014), early warning systems (Pleppen 
& Therriault, 2008) and ninth grade academies (Cook, Fowler, & Harris, 2008). These 
programs focus on the challenges to 9 th grade success and vary in the comprehensiveness of 
their approaches. They also vary on the extent to which they address the four underlying 
theories for student difficulty in 9 th grade. Some focus on monitoring student progress, 
providing accessto supportive adults, preparing students for the transition, or restructuring 
high school into small learning communities. Empirical evidence for the effectiveness of these 
approaches has been modest due to a lack of rigorous evaluation methods that vary from pre 
and posttest designs, quasi-experiments, interrupted time-series, and most recently, a school- 
level randomized controlled trial. 

At the same time, research is emerging on the effectiveness of social emotional 
learning (SEL) interventions and strategies in producing not only increased attendance and 
decreased behavior problems, but also in increasing academic performance. For example, 
Allen, Pianta, Gregory, Mikami, and Lun (2011) conducted an RCT in which teachers were 
given a year of coaching on effective teaching and student-teacher interactions. After a year 
of training, students with teachers in the experimental group scored significantly higher on end 
of the year achievement tests than did students in the teacher control group. Quality of 
student-teacher interaction was a significant mediator of student achievement. 

Overall, SEL strategies that improve academics tend to be carefully planned, theory 
and research based, teach SEL skills for application to daily life, address affective/social 
dimensions of learning, coordinate efforts linked to academic outcomes, address key 
implementation factors, and involve family and community partnerships, continuous 
improvement, and outcome evaluation (Elias et al, 1997). BARR draws on skills developed by 
social emotional learning theory, and focuses on student self- awareness, responsible 
decision-making, relationship skills, social awareness, and self- management. 

Educational, resilience, and developmental research confirms that positive school 
climate, school connectedness, learning engagement, and positive relationships between 
students and staff—and among staff— are essential ingredients for school reform. (Cohen 
2006; De La Ossa 2005; Gordon 2006; Jerald 2006; National Research Council 2004). The 
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degree to which students feel personally connected to their schools has been linked to 
attendance, performance, and graduation (Blum & Libbey 2004; Loukas et al., 2006; 
Wentzel1999). However, positive relationships and a sense of community are not enough to 
produce achievement gains among students without a clear emphasis on academic 
excellence by school staff (Lee & Smith, 1999). Quality pedagogy, caring relationships, high 
expectations, and real-time access to student data are all critical in fostering a positive school 
climate that promotes achievement. 


The Building Assets Reducing Risks (BARR) Model 


The Building Assets Reducing Risks (BARR) model is a comprehensive approach that 
addresses developmental, academic, and structural challenges in the 9 th grade year through 
the implementation of eight different school-wide and individual strategies. It combines 
teachers’ real-time analysis of student data, student asset building, and intensive teacher 
collaboration to prevent course failure. It reaches all students and teachers, and uses SEL 
skills to help increase academic performance. It develops positive student-teacher 
relationships and integrates student supports into a school's existing model for addressing 
non-academic barriers to learning. 

BARR was developed in 1998 by Angela Jerabek, a 9 th grade guidance counselor at 
Saint Louis Park High School in Minnesota. Prior to BARR implementation, the 9 th grade 
course failure rate at St. Louis Park High School ranged from 44 to 47 percent. After one year 
of implementing the BARR model, the failure rate decreased to 28 percent and held steadily 
at 20 percent or lower for the next 15 years. (Evans, Sharma, & Jerabek, 2013). 

BARR is built on three developmental theories: first, developmental assets which are 
forty internal and external sources of support that are critical to young people’s successful 
growth and development (Scales & Roehlkepartain, 2003). The more assets young people 
experience, the more they engage in positive behaviors). Second, risk and protective factors, 
a social development strategy developed by Hawkins and Catalano, that addresses 
substance abuse, delinquency, teen pregnancy, school dropout, and violence (Hawkins & 
Catalano, 2002). Third, the attribution theory of student motivation that articulates the 
cognitive-behavioral-social process by which students develop beliefs about their ability to 
succeed in school (Wentzel & Wigfield, 2009). 
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The BARR model consists of eight strategies that are interconnected and function as a 
whole. These strategies include: 

1) Provide professional development for teachers, counselors, and administrators. 

Professional development focuses on using student-teacher relationships to enhance 
achievement. It begins before the model is implemented and continues throughout the 
school year. This training is conducted by two trainers and one of the trainers is a BARR 
educator. There are also monthly meetings in which the 9 th grade teachers continue to 
receive professional development. 

2) Create cohorts of students. In the BARR model, students take three core courses as 
part of a block, or cohort, of students. Each block has three core-subject teachers 
(typically math, English, and science or social studies), and the teachers’ and students’ 
schedules are aligned so the students take these three core subjects only with other 
students in their cohort. This structure helps educators cultivate connections — with 
students and with each other — that allow for more effective education. For example, in 
a school with an average of 30 students per class, a cohort would include three teachers 
and 90 students. Each of the three teachers (English, math, and social studies, for 
example) would teach three 9th grade sections of his or her class — 30 students per 
class to make up the cohort of 90 students. At times, these teacher groups are joined by 
the BARR Coordinator, Social Worker, 9 th Grade Counselor, and Assistant Principal. 

3) Engage families in student learning. BARR improves communication with families and 
makes them active partners. Families are invited to participate in an initial orientation 
and a parent advisory council. Teachers also regularly call and meet with the parents or 
guardians of students who need more support so the educators and families can work 
together more effectively. 

4) Use BARR’s 1-Time Curriculum to foster a climate for learning. 1-Time is a 30-minute 
weekly lesson that students take with others in their cohort as a supplement to the 
school curriculum. Taught by one or more of the cohort’s three core-subject teachers, I- 
Time’s social/emotional focus helps students build strong relationships with teachers and 
each another — and practice essential life skills, such as communicating effectively and 
setting personal goals. 1-Time also addresses important issues for adolescents, including 
substance abuse, grief, and bullying. 

5) Hold regular meetings of the cohort teacher teams. The three teachers in a cohort 
have the same scheduled planning period. The teacher team meets weekly to discuss 
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each student in the cohort using student-level performance data that is updated weekly. 
The teacher team evaluates each student’s progress as well as academic and non- 
academic obstacles to learning. In these collaborative sessions, teachers identify 
students who are not on track and determine how to intervene to support them. At least 
once a week, the counselor, social worker, BARR coordinator, and assistant principal (or 
administrator in charge of discipline) meet with each group of teachers to facilitate 
review and referrals. 

6) Conduct risk-review meetings. Cohort teacher teams identify persistently low- 
performing students and refer them to a risk review team, which includes the school’s 
BARR coordinator, a school administrator, a school social worker/counselor, and other 
professionals as needed. This team is trained to help the students most in need so the 
highest-risk students get essential external support — and the cohort teachers are able 
to focus on other students in their group. 

7) Focus on the whole student. In every interaction with students (or discussions about 
students), educators address students’ academic, emotional, social, and physical needs. 
Teachers work to better understand and build on students’ strengths — and proactively 
address the non-academic reasons why they may fall behind in school and what they 
need to thrive. 

8) Administrator engagement. Before the school implements the BARR model, 
administrators learn how they can integrate BARR into their school culture and use it to 
make decisions that further their specific school goals. Administrators engage in ongoing 
support, involvement, communication with the BARR team in their school. 

The theories underlying the BARR Model, eight school and student-level strategies, and 
student outcomes are articulated below in the BARR Logic Model. 
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Building Assets Reducing Risks Logic Model 


Frameworks 


School-Level 

Strategies 


Student- 

Level 

Strategies 


Developmental 
Assets 


Risk and 
Resiliency 

Attribution 

Theory 



Student 

Outcomes 




1) Professional 


5) "1-Time," 


Development for 


Curriculum 


Teachers 


6) Block Meetings, 


2) Create a Cohort of 


Collaborative Problem 


Students 


Solving 


3) Contextual Support 


7) Risk Review for 


for Administrators 


Persistently Failing 


4) Engage Families 


Students 




8) Whole Student 
Emphasis 




Use of Real-Time Data 



A. All course credits 
earned 

B. Higher- 

achievement scores 

C. More engaged in 
learning 

D. Reduced adverse 
effects of alcohol and 
other drugs 


This i3 Development grant addressed the following research questions: 

1. What is the impact of one year of participation in BARR on 9 th grade students’ total credits 
earned toward graduation in their core classes, as compared to students who do not 
participate in BARR? 

2. What is the impact of one year of participation in BARR on 9 th grade students’ academic 
achievement in reading, as compared to students who do not participate in BARR? 

3. What is the impact of one year of participation in BARR on 9 th grade students’ academic 
achievement in mathematics, as compared to students who do not participate in BARR? 

4. What is the impact of one year of participation in BARR on 9 th grade students’ credits 
earned, NWEA mathematics and reading scores on more and less proficient students? 
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Chapter III - Impact Study: Within-School Randomized 
Controlled Trial (RCT) of the BARR Model 

The three schools in the i3-supported grant were purposefully chosen to test 
replicability in different geographic locations and in different size schools. Two smaller 
schools in rural Maine and one larger school in southern California participated in the 
grant. In addition, the school where BARR was developed served as a laboratory site 
to test implementation fidelity instruments and refine the BARR strategies. The school 
in southern California was willing and large enough to conduct a within-school 
randomized controlled trial. Those findings are detailed in this section. 

Overall, the results of the RCT study indicated that students in the BARR 
condition earned significantly more core credits, higher grade point averages, and 
higher scores on the spring NWEA mathematics and reading tests compared to 
students in the non-BARR condition. Subgroup analysis revealed that Hispanic 
students in BARR achieved significantly higher spring NWEA mathematics and 
reading test scores than did Hispanic students in the non-BARR condition. Also, 
students with lower academic proficiency who were in the BARR condition earned 
significantly more core credits and higher NWEA mathematics and reading scores 
than did students with lower academic proficiency who were in the non-BARR 
condition. The more academically proficient students in the BARR condition achieved 
significantly higher spring NWEA mathematics scores than did the more academically 
proficient students in the non-BARR condition. 

The following section describes the selection of teachers, random assignment 
of students, student and teacher participants, materials and procedure, data collection 
and analysis, and results. 

Selection of teachers and random assignment of students 

• Selection and assignment of study teachers. Principals assigned teachers to BARR 
or non-BARR conditions. They were told not to select the best or most willing teachers 
to be part of the BARR intervention but to mix teachers as evenly as possible according 
to years of experience, gender, level of education, and ethnicity. Random assignment of 
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teachers in the 9th grade BARR model was not possible due to a number of limiting 
factors such as licensure/credentials and unique electives taught by teachers, as well 
due to the need to balance staff FTE within subject area, which is especially true for 
math and science. 

• Identification of eligible students. The evaluation sample included all students who 
were enrolled in regular core classes (language, math, science, social studies) at the 
school and available to the BARR program. This excluded students in sheltered special 
education classes or sheltered classes for students with very limited English proficiency. 
Ineligible students were identified prior to random assignment. 

• Student assignment procedure. Students were randomly assigned to BARR and non- 
BARR groups by Abt Associates, the i3 oversight evaluators, sorted to ensure balance of 
gender and ethnicity and sent back to the school. Prior to random assignment, the only 
request made by the principal was to ensure that twins or close family members (e.g., 
cousins) be in the same condition, either BARR or non-BARR. This was done to avoid 
community conflict or family perception of differing conditions within the school. In these 
cases, the pairs were randomly assigned by Abt Associates. 

Participants 

Students. The study was conducted in a large suburban high school in southern 
California with a total enrollment of 2,514 students in grades 9-12. A total of 555 9 th grade 
students, 54% female and 46% male, participated in the study. Students educated in sheltered 
instruction were excluded from participating. Racial composition was 52% Caucasian, 37% 
Hispanic, and 11% African American, Asian, American Indian, or mixed races. Sixty-eight 
percent of students were eligible for the free or reduced price lunch program, and 17% were 
ELL students. 

To form the groups of more or less proficient students, we computed the median for 
the fall reading and mathematics Northwest Evaluation Association (NWEA) scores. Students 
at or above the median on both tests were classified as "more proficient" (n= 203) and those 
below the median on either one or both tests were classified as "less proficient" (n= 318). 

Teachers. Table 1 lists the demographics of teachers participating in the BARR impact 

study. 
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Table 1. BARR Teacher Demographics at Hemet High School during Randomized Controlled 
Trial (School Year 2011-2012) 



Number 

Female 

Male 

Caucasian 

Hispanic 

Other 

Experience 

BA/BS 

MA/MS 

BARR 

9 

6 

3 

9 

0 

0 

1 3 yrs. * 

2 

7 

Non-BARR 

17 

14 

3 

16 

1 

0 

1 3.5 yrs.** 

2 

15 


*Mean number of years, SD=8.23; ** Mean number of years, SD=8.52 


There were no statistically significant differences between BARR and Non-BARR 
teachers on any of the demographic variables. In terms of sample size, there were almost twice 
as many Non-BARR teachers (n=17) as there were BARR teachers (n=9). This is due to the 
restructuring that is part of the BARR Model (affecting the BARR side) but which did not affect 
the Non-BARR side. BARR blocks in this year consisted of English, Mathematics, and High 
School 101 subject areas. Science was not blocked during this year, due to the large number of 
science subjects taught in 9th grade; therefore, science teachers are not included in Table 1. All 
teachers on both the BARR and non-BARR side were credentialed in their subject areas. 

Materials and Procedure 

The BARR model was implemented with the experimental group during school year 
2011 -2012. The non-BARR group was “business as usual.” 

• A site coordinator was chosen by the principal to help carry out implementation of the 
BARR model and to act as a liaison between the school and the BARR technical 
assistance provider and evaluators. The site coordinator serves as a key conduit of 
information (i.e. failure rate trends, policy impacts/revisions) between the BARR teams 
and principal which provides contextual support and leads to systemic change. The 
coordinator is also tasked with implementing recommended improvements provided by 
BARR technical assistance providers in order to achieve and maintain high fidelity. 

• Teachers and administrators received two days of training on developmental assets and 
the BARR model. 

• To assist with implementation of the eight BARR strategies, a technical assistance 
provider called the BARR coordinator at least weekly. In addition, the project director 
and BARR coordinator from St Louis Park High School conducted monthly Professional 
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Learning Community conference calls with the coordinators and principals from all the 
schools, including the non-RCT schools. 

• The importance of random assignment and maintaining the integrity of the design was 
emphasized to the principal and site coordinator. On a regular basis, the site coordinator 
shared with the technical assistance provider information about any changes of status in 
study students. Change of status might occur, for example, when a student left the 
district, was provided a Special Education plan, or was moved to a different level of 
English language proficiency. 

• BARR developed a system that classified all students as Level 0,1,2, or 3, in order to 
identify, communicate and intervene with students who are in need of support to prevent 
academic failure and accelerate students to achieve at their highest potential. This 
common language works to align school and community wide resources which improves 
the efficiency of student intervention and support. Level 0 are thriving students: Students 
passing all courses, socially-emotionally resilient, may occasionally need additional help 
to do their best and classroom teacher provides this support or student recovers on their 
own. Level 1 are brief intervention students - Students are failing one or more courses 
for two weeks or more during the quarter. They are socially-emotionally typical of other 
high school students. Student not responding to Brief Intervention are advanced to 
Level-ll. Level II are Students are still failing after Brief Intervention. They may have 
social, behavioral, or mental health problems affecting school progress. If students 
continue to fail and have persistent and serious social, behavioral, or mental health 
problems during Team Intervention, the site coordinator asks Risk Review Team 
(Counselor/SW) members if the student should be enrolled in their review. Level III - 
Risk Review intervention - Students not successful in Team Intervention and/or students 
require immediate evaluation for referral to community resources. They are followed 
after Risk Review Intervention in Risk Review meetings. 

Data Collection and Analysis 

Data were collected at the end of each semester on credits earned and NWEA 
achievement scores. NWEA tests were administered by the school at the beginning and end of 
the academic year. Each school was assigned a window of time during the beginning of the 
school year in which to administer this test (window determined by the national center). 
Typically, testing occurred in October and May. 
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Separate OLS regression analyses were conducted to predict total core credits 
earned and spring NWEA reading scores. For both of these analyses, Study Group, Gender, 
Hispanic origin, and fall NWEA reading scores served as predictor variables. An additional 
OLS regression analysis was conducted to predict spring NWEA mathematics scores, using 
Study Group, fall NWEA mathematics scores, Gender, and Hispanic origin as predictor 
variables. We did not impute missing data. Since our analyses consisted of three univariate 
models we only included students who had data on that particular outcome. 

Results 

Attrition. 

Tables la, 1b, and 1 c in Appendix A list the attrition rate for credits earned, 
NWEA reading scores, and NWEA mathematics scores. Overall attrition, as well as 
differential attrition were low and within acceptable standards. For credits earned, 
overall attrition was 1.3% and differential attrition was 1.8%. For NWEA reading, 
overall attrition was 10.8% and differential was 9.4%. For NWEA mathematics, overall 
attrition was 8.7% and differential attrition was 7.3%. 

Core Credits Earned 

Tables 3a, 3b, and 3c in Appendix C present the mean number of core credits earned 
and mean NWEA reading and mathematics scores for fall and spring by Study Group, Gender, 
and Hispanic origin. Figure 1 displays the mean credits earned by BARR and non-BARR 
groups. 


15 






Building Assets Reducing Risks 


Core Credits Earned 



Figure 1. Core credits earned by study group 

Table 4a in Appendix D displays the results of the regression analysis that predicts 
mean number of core credits earned. The model significantly predicted 15.8% of the variability 
in core course credits earned (F (4,516) = 24.205, p<0.001). Study Group was a significant 
predictor, as was fall NWEA reading scores. Students in the BARR experimental group earned 
significantly more core course credits (M=5.65) toward graduation than students in the non- 
BARR group (M=5.39). Neither gender nor Hispanic origin was a significant predictor of core 
credits. 

Grade Point Average 

In addition, we analyzed Grade Point Averages (GPAs) between students in the BARR 
and non-BARR conditions. While GPA as an outcome measure is subject to criticism because 
of its subjective nature, we nonetheless examined GPAs as a measure of quality of work. 
Students in the BARR treatment group earned significantly higher cumulative GPA in core 
courses (average GPA=2.91) as compared to students in the non-BARR group (average 
GPA=2.67) by the end of their 9 th grade year (t=2.729, df = 533, p=.007, effect size = .24). The 
results held true when examining fall semester GPAs (t=2.566, df = 546, p=.011, effect size 
=.24) and spring semester GPAs (t=2.924, df = 533, p=.004, effect size = .25). 

Spring NWEA Mathematics Scores 

A second measure of academic progress was growth in achievement scores on NWEA 
standardized tests of reading and mathematics administered in fall and spring to all 9 th -grade 
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students. Table 4b in Appendix D displays the results of the regression analysis to predict 
spring NWEA mathematics scores and Figure 2 presents the results graphically. The model 
significantly predicted 77% of the variability in spring NWEA mathematics scores (F (4,470) = 
388.118, p<0.001). 

Study Group was a significant predictor, as was fall NWEA mathematics scores. On 
average, students in the BARR experimental group improved 6.45 points while students in the 
non-BARR group improved 1.14 points. This translates into an improvement of two grade levels 
for the BARR experimental group (8th grade to 10th grade equivalent) compared to the loss of 
one grade level (8th grade to 7th grade equivalent) for the non-BARR group. Gender was not a 
significant predictor of spring NWEA mathematics scores, but Hispanic origin was significant 
with non-Flispanic students earning higher spring NWEA mathematic scores than Hispanic 
students across experimental conditions. 



Spring NWEA Reading Scores 

Table 4c in Appendix D displays the results of the regression analysis to predict spring 
NWEA reading scores and Figure 2 displays the results graphically. The model significantly 
predicted 67% of the variability in spring NWEA reading scores (F (4,490) = 251.841, 
p<0.001). Study Group was a significant predictor, as was fall NWEA reading scores. On 
average, students in the BARR experimental group improved 4.51 points while students in the 
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non-BARR group improved 2.70 points. Both groups scored above grade level in the fall and 
continued to score above grade level in the spring. Gender was not a significant predictor of 
spring NWEA reading scores, but Hispanic origin was significant with non-Hispanic students 
earning higher spring NWEA reading scores than Hispanic students across experimental 
conditions. 

Table 2 below lists the grade equivalents for NWEA scores for BARR and Non-BARR 
students in fall and spring test administrations. 

Table 2. Grade Equivalents for NWEA Scores by condition 



Reading 

Mathematics 


Fall 

Spring 

Fall 

Spring 

Non-BARR 

9.5 

11 + 

8 

7 

BARR 

10 

11 + 

8 

10 


Hispanic Ancestry Subgroup 

Table 3c in Appendix C lists descriptive statistics by Hispanic Ancestry. An OLS 
regression (see Tables 4b and 4c in Appendix D) revealed that across treatment conditions, 
Hispanic students scored significantly lower on spring NWEA mathematics tests (t(2,480) 
=2.842, p=0.013) and reading tests (t(2,493)=-2.601, p=0.010) than did non-Hispanic 
students. 

However, when analyzed according to treatment condition, Hispanic students in the 
BARR condition achieved significantly higher spring NWEA mathematics (t (2,183) = 2.740, 
p=0.007) and Reading (t (2,187) = 2.066, p=0.040) scores than did Hispanic students in the 
non-BARR condition. Core credits and GPA were also higher for Hispanic students in the 
BARR condition than for Hispanic students in the non-BARR condition, but differences were 
not statistically significant. 

Non-Hispanic students in the BARR condition earned significantly more credits (t (2, 
343) = 3.72, p = 0.001), and achieved higher NWEA Mathematics scores (t (2,317) = 2.447, 
p=0.015) than did non-Hispanic students in the non-BARR group. Reading scores and GPA 
were higher for students in BARR compared to those in the non-BARR condition, but the 
differences were not statistically significant. 

Figures 3 and 4 display differences between Hispanic and non-Hispanic students by 
study group for both NWEA mathematics and reading scores. 
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Figure 3. NWEA mathematics scores by Hispanic ancestry 


NWEA Reading Scores by Hispanic Ancestry 
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Figure 4. NWEA Reading scores by Hispanic ancestry 


Academic Proficiency Subgroups 


Students in both the BARR and Non-BARR groups were separated into two groups 
based on scores they obtained on NWEA reading and mathematics tests in the fall. Less 
Proficient students were defined as those who scored below the median on either the NWEA 
mathematics or reading tests. More Proficient students were defined as those who above the 
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median on both the NWEA mathematics and reading tests. Table 2 in Appendix B lists the 
descriptive statistics for each group. Separate OLS regression analyzes were conducted by 
proficiency group for each of the outcome measures (see Appendix F, Tables 6a-6f). 

As displayed in Figure 5, Less Proficient students in the BARR condition earned 
significantly more credits in core classes compared to Less Proficient students in the non- 
BARR condition (p=0.020, effect size = .29). More Proficient students in the BARR condition 
also earned more core credits than More Proficient students in the non-BARR condition, but 
this difference was not statistically significant. 



Core Credits Earned by Proficiency Group 

5.88 



LESS MORE 


■ BARR 

■ CONTROL 


*p<.05 


Proficiency Group 


Figure 5. Core credits earned by proficiency group 


In terms of scores achieved on the NWEA reading test, Less Proficient students in the 
BARR condition earned significantly higher spring test scores than their Less Proficient peers 
in the non-BARR condition (p=0.005, effect size = 2.49). More Proficient students in the 
BARR condition also achieved higher test scores than More Proficient students in the non- 
BARR condition, but this difference was not statistically significant (see Figure 6). 
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Spring NWEA Reading Scores by Proficiency 
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Figure 6. Spring NWEA reading scores by proficiency group 


Students in both the More and Less Proficiency groups who were in the BARR 
condition earned significantly higher NWEA spring mathematics scores than students of both 
proficiency groups in the non-BARR condition (see Figure 7). BARR students who were Less 
Proficient outperformed their Less Proficient non-BARR peers (p=0.001, effect size=4.24) and 
BARR students who were More Proficient outperformed their More Proficient non-BARR 
peers (p=0.001, effect size=6.24). 
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Spring NWEA Math Scores by Proficiency 
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Figure 7. Spring NWEA mathematics scores by proficiency group 
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After the RCT: BARR in Years 2 and 3 

After the RCT year, the school chose to continue BARR and implement it with the entire 
freshman class. We continued to collect impact and implementation fidelity data in years 2 and 
3 of the project. In addition, we developed a new metric, student failure rate (the percentage of 
students who failed one or more courses at the end of the 9th grade), which was a useful 
measure for schools. At the end of the RCT year in year 1, the student failure rate was 31.9% 
for non-BARR students and 21% for BARR students. After full implementation in the 9 th grade, 
failure rate continued to decrease and by year 3, the failure rate was down to 18.5% for all 
students; a decrease of 42% from before BARR was introduced. The quality of student work as 
measured by average GPA continued to improve. In year 1, the average GPA was 2.67 for 
non-BARR students, and by year 3 had increased to 2.76 for all students. The fidelity of 
implementation of all aspects of the BARR model demonstrated continual growth and reached a 
combined quality of implementation score of 93% by the end of year 3. 

Table 3 presents average number of core course credits earned at the end of freshman 
year, number of failed core classes, failure rate as defined by the number of students with at 
least one failure in a core course, and Grade Point Average (GPA) for three cohorts of freshman 
classes. Results are presented for BARR and non-BARR groups separately in year 1. In years 
2 and 3, all 9 th grade students received the BARR model. 

Table 3. Core Course Credits 


Year 

Number 

of 

students 

Average number of core 

credits earned* 

Number of failed 

core courses 

Percent of 
students with at 
least one failure 
in a core course 

Average GPA in 

core courses 

Year 1 

Non-BARR 

276 

5.26 (87.7%) 

181 

31.9% 

2.67 

Year 1 

BARR 

272 

5.65 (94.2%) 

90 

21.0% 

2.91 

Year 2 

517 

5.63 (94.5%) 

192 

17.2% 

2.70 

Year 3 

530 

5.56 (92.7%) 

231 

1 8.5% 

2.76 


* Average number of attempted core credits was 6.00. 


Figure 8 displays the student failure rate for years 1-3. By the end of year 1, non- 
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BARR students had a failure rate of 32%, compared to a failure of 21% for students in the 
BARR group. By years 2 and 3, the failure rate for all 9th grade students continued to decline 
to between 17-18%. 



Figure 8. Student failure rate across three cohorts of BARR implementation 


During the three years of BARR implementation in the freshman year: 

• Percentage of core credits earned increased from 87.7% for non-BARR students to 92.7% 
for all students. 

• Number of failed core courses decreased by 50% in year 1. 

• In year 1, the student failure rate was 31.9% (for non-BARR students) and by year 3, it 
was down to 18.5% for all students; a decrease of 42%. 

• Average GPA for the BARR students in year 1 and for all students in subsequent years 
was consistently higher than the non-BARR average GPA in year 1 

Academic Performance by Gender 

Table 7a in Appendix G presents the student failure rate by gender across three cohorts 
of 9 th grade students, and Figure 9 presents these results graphically. 


23 















Building Assets Reducing Risks 


Student Failure Rate by Gender across Three Cohorts 



Figure 9. Student failure rate by gender across three cohorts of BARR implementation 
During the three years of BARR implementation in the freshman year: 

• Over time, percentage of core credits earned by boys increased 5 percentage points to 
90% (85.2%, year 1 non-BARRs to 90.0%, year 3) and 6 percentage points for girls 
(90.0%, year 1 non-BARRs to 96.1%, year 3). 

• Failure rate decreased from 38.3% to 24.9% for boys, and from 26% to 10.5% for girls 
from year 1 non-BARRs to year 3. This difference was statistically significant in year 3 in 
favor of girls, i.e., girls had a statistically significantly lower failure rate than boys. 

• Average GPA did not change significantly for boys, but improved for girls from 2.82 to 
3.05 from year 1 non-BARRs to year 3. 


Closing the Achievement Gap 

Of particular note in years 2 and 3 was the decline in student failure rate for Hispanic 
students and closing of the achievement gap between Hispanic and non-Hispanic students. 
Failure rates for Hispanic students went from 41.6% in year 1 for students in the non-BARR 
group to 21.1% for all Hispanic students in year 3; a reduction of 50%. By the end of year 2, the 
failure rate for Hispanic students was no longer statistically significant from the failure rate of 
non-Hispanic students. This trend continued in year 3. In addition, the average GPA earned by 
Hispanic students increased from 2.44 for non-BARR students in year 1 to 2.75 for all Hispanic 
students in year 3. 
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Table 7b in Appendix G lists the average number of core credits earned, failed core 
classes, failure rate, and GPA in core courses. Figure 10 compares the student failure rate for 
Hispanic and non-Flispanic students across the three years of BARR implementation. By the 
end of year 1, a significant achievement gap was evident between Hispanic and non-Flispanic 
students in both the BARR (X 2 ( i, n =27o= 5.608, p<. 02 ) and non-BARR group (X 2 ( i, n = 251 ) = 5.331, 
p<. 02 ). However, by year 2, the gap had closed to reveal an 18% failure rate for Hispanic 
students and a 16% failure rate for non-Hispanic students. This trend continued into year 3, 
with both years showing no difference in failure rate between Hispanic and non-Hispanic 
students. 


Student Failure Rate by Hispanic Origin across Three Cohorts 



Figure 10. Student failure rate by Hispanic origin across three cohorts of BARR implementation 
During the three years of BARR implementation in the freshman year: 

• Percentage of core credits earned by Hispanic students increased from 85.2% (non- 
BARR group) in year 1 to 92.5% (all Hispanic students) by year 3. 

• In year 1, Hispanic students in BARR had a lower failure rate than Hispanic students in 
the non-BARR group. Overall, though, Hispanic student failure rate was higher than 
non-Hispanics in both BARR and non-BARR groups. 

• By year 3, Hispanic failure rates went from 41.6% (non-BARR) to 21.1% (BARR) a 50% 
reduction. The failure rate for Hispanic students was no longer significantly different 
from the failure rate of non-Hispanic students in Years 2 and 3. 
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• Hispanic students' average GPA increased from 2.44 for non-BARR students in year 1 to 

2.75 in year 3. 

Several aspects of the BARR Model provide an explanation for this closing of the 
achievement gap with Hispanic students. First, BARR begins with the individual student and 
provides a forum for multiple adults to know them. Through the effective use of data there is a 
transparency of who is doing well, who is struggling, and what their barriers may be both 
individually and collectively. BARR teacher teams adjust pedagogy to meet the differentiated 
needs of each student. This is core to closing the achievement gap. 

Second, within teams, teachers discuss the macro and micro impacts of race and 
ethnicity with their colleagues. Block Meetings and Risk Reviews provide a safe and effective 
structure for teachers/staff to discuss sensitive issues, learn to address them with students, and 
scale the most effective strategies school-wide. This support enables teachers/staff to 
overcome biases, fears, and stereotypes and see students for who they are, which results in 
more effective teaching and closing the gap. 

Third, The 1-Time curriculum develops a sense of valuing and belonging that reduces 
fear in minority students that they will be stereotyped according to their race or 
ethnicity. Research reveals that once fear is reduced, students can focus on schoolwork, do 
better in classes, and form expectations of future success (Cohen et al, 2009). 

Fourth, Interventions built on positive student teacher relationships encourage and 
support all students. Discussing each student on a weekly basis enables teachers who share 
the same students to identify, attempt, and scale strategies to maximize collective impact. The 
intentional execution of culturally appropriate 1-Time lessons are included as one of the BARR 
Fidelity Behavioral Anchors. 

Fifth, data on established levels of concern based on grades, attendance, and socio- 
emotional issues are applied systemically across the school to ensure that ALL students in need 
of support receive it and are prioritized to ensure maximum impact. Data drive the intervention 
and action plans which leads to a less political and more pragmatic result-oriented environment 
to close the achievement gap (Corsello, Jerabek, & Barbeau, 2015). 

Sixth, at the beginning of Year 2, all teachers and counselors received a one-day 
training specific to equity issues and implementation of BARR. 
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Chapter IV - Measuring Implementation Fidelity 

Fidelity of implementation is key to understanding the impact of the BARR Model. By 
the end of year 3, only two of the schools met the threshold for implementation fidelity using the 
Quality Assessment Rating forms. The school that did not reach implementation fidelity had 
higher student failure rates and lower teacher endorsement of the BARR strategies. 

Implementation Fidelity in the Study Schools 

Creation of implementation fidelity forms and data collection 

During the first two years of the grant, we developed, tested, and refined four measures 
of implementation fidelity. One measure, conducted through an extensive interview with the site 
coordinator, measured the extent to which the 8 components of the BARR model were 
implemented in the school. The other three measures, conducted through evaluator 
observation of Block Meetings, Risk Review, and I Time, measured the quality of 
implementation of each of those components. In addition, we developed a teacher survey to 
measure the extent to which teachers felt that the BARR Model improved their effectiveness as 
teachers. In terms of the implementation fidelity forms, we followed the process described 
below. 

• We drafted a fidelity metric to assess the extent and nature to which the 8 BARR 
strategies have been implemented in each school. We presented this metric to the 
schools in a collective meeting to obtain their input, and refined the metric accordingly to 
reflect the reasoned judgment and real-world experience of school staff and the BARR 
coordinator in each school. The BARR Strategy Implementation Review form served as 
our main measure of implementation fidelity. 

• In addition to the BARR Strategy Implementation Review form, we developed three 
observation forms to gather data on the quality of implementation of Block Meetings, 

Risk Review Meetings, and I Time curriculum. 

• We conducted site visits, each lasting 1.5 days, twice a year at each school during the 
three years of BARR implementation. During these site visits, we observed block 
meetings, Risk Review, and I Time. We also conducted interviews with the 
Superintendent, Assistant Superintendent, Principal, and Assistant Principal twice 
annually. At the end of each site visit, we met with each coordinator and arrived at a 
score for each school on each of the 8 strategies using the BARR Strategy 
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Implementation Review form. In addition to our own observations and interview data, we 
asked each coordinator for specific examples to substantiate the rating that was 
assigned. 

• With this input from schools and project staff, we established a threshold for adequate 
fidelity for each strategy and summed across strategies. This was done using our 
theoretical knowledge of the program and judgment of minimal level required for 
adequate fidelity, as well as the realistic, empirical data obtained from these 
underperforming schools implementing the program for the first time. Using these two 
sources of information, we determined strategy-specific and program-level threshold 
scores. Abt Associates, our oversite evaluators, notified the BARR program met the 
Government Performance and Results Act (GPRA) standard for generating high-quality 
implementation results. Appendix H shows the thresholds for each key component, and 
the determinations of fidelity for each key component for the first two years using the 
BARR Strategy Implementation Review form. 

• All of the fidelity forms were revised at the end of the grant period to include additional 
detail and precision in assessing implementation fidelity. We found that our initial forms 
were not able to detect lack of engagement and negative interactions between students 
and teachers, and between students, resulting in inflated implementation scores in some 
cases. 

In addition to these measures, we developed a teacher survey to assess the degree to 
which teachers reported that the BARR strategies impacted various components of their 
teaching effectiveness. 

Implementation Results - BARR Strategy Implementation Review 

The BARR Strategy Implementation Review measures each of the 8 key components 
(strategies) of the BARR program through an in-depth interview with the site coordinator. 

During the interview, the site coordinator assigns a score from 1 to 9 for each of the 8 key 
components. The total range possible, summing across the 8 strategies, is 8 to 72. Figure 11 
shows a graphical representation of BARR Strategy Implementation for the three replication 
schools in years 2011 - 2014. 
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Figure 11. BARR Strategy Implementation Interview results 


Observations of Block Meeting, I Time, and Risk Review 

In addition to the BARR Strategy Implementation form interview, we observed a sample of Block 
Meetings, Risk Review Meetings, and I Time classes. 

• BARR Block Meeting Process Quality Assessment. This observation form measures 
application of enrollment guidelines, consensus problem solving, identification of root 
causes, goal setting, identification of barriers and assets, development of promising 
solutions, and implementation of an action plan. 

• BARR 1-Time Process Quality Assessment. The observation form measures teacher 
mindset emphasizing student growth mindset, classroom management, assets as the 
focal point of the discussion, recognition of students’ unique assets, recognition of 
student relationships, recognition of student challenges, recognition of student values, 
and classroom spirit. 

• BARR Risk Review Process Quality Assessment. Measures on the observation form 
include application of enrollment guidelines, best practices for students in emerging 
crisis, best practices for engaged continuing issue students, best practice for unengaged 
continuing issue students, consensus problem solving, identification of root causes, goal 
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setting, identification of barriers and assets, development of promising solutions, and 
implementation of an action plan. 

Summary of Observational Quality Assessment Ratings 

In order to have one measure that represents the quality of BARR implementation, we 
calculated an overall summary of the observational ratings in the spring of each of the grant 
years in each school by combining all the observational scores into one average. The average 
was based on the percentage of points achieved in each component based on a total of 100%. 
Figure 12 presents this information. 


Block, Risk, and I Time Observation of Quality 



Figure 12. Block, Risk, and I Time observation of quality 

All three schools achieved implementation fidelity on the BARR Strategy Implementation 
Interview indicating that all BARR structures were in place. Flowever, only two schools 
achieved an implementation fidelity threshold of 80% on the quality of implementation based on 
a combination of the observational data from Block Meetings, I Times, and Risk Review. 
Revised forms will provide a more precise measurement of implementation fidelity that better 
represents the full range of responses in both structural and quality measures. 

Relationship of Implementation Fidelity to Student Failure Rate 
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We then compared the core course failure rate in each school at the end of each year of 
implementation. Failure rate is defined as the percentage of students with one or more F’s in a 
core course. The schools that improved in quality of implementation also had improved core 
course failure rates, while the school that did not improve in quality of implementation had a 
failure rate that increased. A Pearson correlation coefficient calculated between quality 
assessment observation scores and failure rate was -0.380, and although in the expected 
direction, was not significant (p >.20, df = 7). Individually, growth in implementation quality in I 
Time and Block Meeting were negatively correlated with failure rates, but again not significantly, 
perhaps due to the small sample (n=9 ratings). Overall, observations of implementation quality 
were more strongly related to student outcomes (i.e., core course failure rate) than were 
interview ratings that the BARR structural elements were in place (Sharma, Corsello, & Jerabek, 
2014). 

Feedback from BARR Teachers 

As an additional part of the grant, we conducted a survey with 9 th grade BARR teachers 
to gather their opinions about the BARR model, their relationships with students, and any impact 
that BARR may have had in improving their effectiveness as a teacher. The survey focused on 
the BARR strategies, with additional questions on the Block Meeting. There were a total of 19 
questions that were rated on a 5 point Likert scale of strongly disagree to strongly agree. 
Appendix I contains the percentage of responses for each of the items. 

A total of 46 staff members from Schools A, B, C, and D (the school where BARR was 
developed that served as a laboratory for testing new instruments and strategies) completed the 
survey through SurveyMonkey.com in June through August 2013. The number of respondents 
varied according to school size, with School A (13), School B (5), School C (18), and School D 
(10) respectively. Teachers indicated the number of years they have been teaching, with almost 
one third reporting they have taught for 6-10 years (see Figure 13). 
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Reported Years Teaching 


■ 1-2 Years 

■ 3-5 Years 

■ 6-10 Years 

■ 11-15 Years 

1 6 or more Years 


Figure 13. Reported years of teaching 

Results 

The categories of strongly agree and agree were combined to provide an overall 
endorsement of items. Results indicated that over 90% of teachers reported that they felt proud 
of their team’s accomplishments for students, liked the opportunity to share ideas about how to 
solve student learning problems, thought that the use of data on a weekly basis enabled them to 
support students in real time, and got helpful ideas about student-teacher relationships from the 
team. 


13 



Years Teaching 


In addition, 89% of teachers reported feeling less isolated, learning new ideas from 
other teachers on how to solve student problems, had higher morale at school, felt a sense of 
shared as opposed to sole responsibility for students with behavior problems, felt they had a 
team supporting them when a student was in real trouble, and had stronger bonds with other 
teachers. 

Responses were analyzed by years of teaching. There were no significant 
differences in ratings of the BARR model by years of teaching. Responses of new teachers 
(those teaching 1 to 2 years) were not significantly different from responses of veteran teachers 
(those teaching 16 or more years). 

Responses were also analyzed according to whether schools in this grant period 
achieved implementation fidelity using the quality assessment measure. Schools that met 
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implementation fidelity were combined (Schools A and B, n=18 teachers) and compared with 
the school that did not meet implementation fidelity (School C, n=18 teachers). Teachers in 
schools that achieved implementation fidelity rated all six program-related questions significantly 
higher than did teachers in the school that did not meet implementation fidelity. Specifically, 
these questions included: use of I Time improved relationships with students, use of data 
enabled teachers to support students in real time, training in the developmental assets has 
improved teacher perception of student strengths, referral to Risk Review enabled teacher to 
focus more effectively on all of their students, use of Block Meeting has increased openness 
and effectiveness of communication between teachers and administrators, and educational 
technical assistance has increased their effectiveness (p values for these items ranged from 
0.013 to 0.001 when testing group differences). 

This pattern of teacher self-report on BARR strategies was consistent with evaluator 
observation of the quality of BARR implementation. Evaluator observation of quality and 
teacher self-report helped to provide differential assessment of schools that implemented BARR 
with fidelity and those that did not. 
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Chapter V - Impact Study: BARR in 2 Rural Schools 

The BARR Model was implemented in two small rural high schools in Maine. Results 
indicated that School B, that implemented BARR with fidelity, had student core course failure 
rates that continued to decline and scores on the NWEA mathematics and reading tests that 
continued to increase. 

An RCT was attempted in the larger of the rural schools (School C). However, 
contamination of the BARR strategies and practices became an issue, and the RCT was 
discontinued prior to the end of the first grading period. The school continued to receive 
services and we tracked implementation fidelity and student outcomes. After three years, 

School C did not reach implementation fidelity, student core course failure rates did not improve, 
and problems in NWEA test administration made test scores difficult to interpret. 

High School B 

School B is a small rural high school on the coast of northern Maine with a total 
enrollment of 302 students in grades 9-12. School B has 30 full-time teachers and 2 
administrators, 34% of students are eligible for free or reduced lunch, and 96% are Caucasian. 

School B has shown increasing fidelity of implementation over the three years of the 
BARR program, resulting in a final combined quality observation score of 90%. Over the past 
three years, School B has shown growth in the student academic outcomes of credits earned in 
core courses during the freshman year and in performance on the NWEA standardized tests. 

Core Course Credits 

Table 4 presents the average number of core course credits earned at the end of 
freshman year, number of failed core classes, failure rate as defined by the number of students 
with at least one failure in a core course, and Grade Point Average (GPA) for three cohorts of 
freshman classes in school years 2011 - 2014. 
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Table 4. Core Course Credits 


Year 

Number of 
students 

Average 
number of 
credits 
earned* 

Number of failed 

core courses 

Percent of students 
with at least one 
failure in a core 

course 

Average GPA in core 

courses 

Year 1 

78 

3.51 (92.6%) 

22 

16.7% 

82.17 

Year 2 

62 

3.97 (87.0%) 

31 

1 8.0% 

83.60 

Year 3 

69 

3.81 (95.3%) 

13 

1 1.6% 

85.65 

* Average number of attempted credi 

ts was 3.79 in Year 

, 4.56 in Year 2, and 4.00 in Year 3. 


During the three years of BARR implementation in the freshman year: 

• Percentage of core credits earned increased from 92.6% to 95.3% 

• Number of failed core courses decreased by 41 % (from 22 to 13) 

• Failure rate decreased by 31 % (from 16.7% to 11.6%) 

• Average GPA improved steadily from 82% to 86% 

These results demonstrate that the implementation of the BARR model positively affected 
9 th grade student achievement over time at School B and that these improvements have been 
sustained. 

Northwest Education Association (NWEA) Mathematics, Reading, and Language Scores 

A second measure of academic progress was achievement scores on NWEA 
standardized tests of reading, mathematics, and language administered in fall and spring to all 
9 th grade students each year. 

In the first year of BARR implementation, School B students scored below the national 
average in reading (8th grade equivalent in fall and spring). In years 2 and 3, School B students 
scored above the national average, and by year 3, School B students were scoring greater than 
11 th grade equivalent in both fall and spring (n=71 students in year 1, n=47 students in year 2, 
and n=64 students in year 3). Figure 14 presents these results. 
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Figure 14. Growth in NWEA Reading scores by year as compared to national average 

In year 1, School B students were well below the national average in mathematics 
(scoring at grade 7.5 in both fall and spring). This gap closed by spring of 2013 when School B 
students scored at the national average for 9th grade. In year 3, students scored at a grade 8.5 
equivalent. Overall, performance in mathematics was weaker than in reading and language, but 
progress is, nonetheless, evident. By year 3, School B students were 0.5 years behind the 
national average compared to 1.5 years in the first year of the BARR model (n=74 students in 
year 1, n=48 students in year 2, and n=65 students in year 3). Figure 15 presents these results. 



Figure 15. Growth in NWEA Mathematics scores by year as compared to national average 
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In Years 2 and 3, School B students took the NWEA language test, in addition to reading 
and mathematics. In the fall of year 2, School B students scored at the national average and in 
spring of year 1 and the following year, they scored greater than 11th grade equivalent (n=45 
students in year 2 and n=67 students in year 3). Figure 16 presents these results. 



Figure 16. Growth in NWEA Language scores by year as compared to national average 

Table 5 shows that freshmen students at School B increased their performance on 
NWEA reading tests by three grade levels (grade 8 to 11+), and on language tests by two grade 
levels from 2011 to 2014. In mathematics, School B students gained one grade level from the 
first to third years of the grant. 


Table 5. Grade Equivalents for NWEA Scores by Year 



Reading 

Mathematics 

Language 


Fall 

Spring 

Fall 

Spring 

Fall 

Spring 

Year 1 

8 

8 

7.5 

7.5 

N/A 

N/A 

Year 2 

9.5 

11 + 

8.5 

9 

9 

11 + 

Year 3 

11 + 

1 1 + 

8.5 

8.5 

1 1 + 

1 1 + 
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High School C 


School C is a rural high school in southern Maine with a total enrollment of 1,018 
students in grades 9-12. School B has 78 full-time teachers and 5 administrators, 44% of 
students are eligible for free or reduced lunch, and 95% are Caucasian. 

As mentioned previously, School C attempted an RCT. However, because of 
contamination of BARR strategies and practices across BARR and non-BARR groups the study 
was discontinued prior to end of the first marking period. After that, the original BARR group 
continued to receive technical assistance which was then shared by the site coordinator and 
counselor with the non-BARR group. After three years, many problems still remained in the 
degree to which School C implemented the BARR Model. Student outcomes indicated no 
improvement in core course failure rate and uneven performance on NWEA tests of reading and 
mathematics, sometimes due to problems in test administration. 

Core Course Credits 


Table 6 below presents the average number of core course credits earned at the end of 
freshman year, number of failed core classes, failure rate as defined by the number of students 
with at least one failure in a core course, and Grade Point Average (GPA) for three cohorts of 
freshman classes in school years 2011 - 2014. 

Table 6. Core Course Credits 


Year 

Number of 
students 

Average 
number of 
credits 
earned* 

Number of failed 

core courses 

Percent of students 
with at least one 
failure in a core 

course 

Average GPA in core 

courses 

Year 1 

220 

4.40 (89.2%) 

1 17 

26.8% 

83.2 

Year 2 

238 

4.05 (83.0%) 

160 

33.6% 

79.6 

Year 3 

223 

3.96 (83.4%) 

176 

33.2% 

80.9 


Average number of attempted credits was 4.93 in year 1,4.89 in year 2, and 4.75 in year 3. 


During the three years of BARR implementation in the freshman year: 


• Percentage of core credits earned decreased from 89.2% to 83.4% 

• Number of failed core courses increased by 50% (from 117 to 176) 

• Failure rate increased by 24% (from 26.8% to 33.2%) 
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• Average GPA decreased from 83.2% to 80.9% 

These results demonstrate an overall negative trend in student academic outcomes. 

Northwest Education Association (NWEA) Mathematics, Reading, and Language Scores 

A second measure of academic progress was achievement scores on NWEA 
standardized tests of reading, mathematics, and language administered in fall and spring to all 
9 th grade students each year. Figure 17 presents the results for NWEA reading scores. Results 
were variable from year to year, but the trend over 3 years shows a decline from a high of 11n- 
grade equivalent in year 1 to just below 9th grade equivalent in year 3. Of note is that 80% of 
students took the test in year 1,89% of students in year 2, and 92% of students in year 3. 



Figure 17. Growth in NWEA Reading scores by year as compared to national average 

In terms of performance in mathematics, students scored at the 9th grade national 
average in mathematics in the fall of year 1, and improved two grade levels in the spring of that 
year to score at 11th grade equivalent. A total of 95% of students completed both fall and spring 
tests. Fiowever, in year 2, only 27% of the freshman class completed both tests, and in year 3, 
only 55% completed these tests. Given the much reduced sample sizes in years 2 and 3, these 
results could not be generalized to the entire 9 th grade class. 
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In year 3, 93% of students took the NWEA language test. School C students scored one 
half year above the national average in the fall and one year above the national average in the 
spring. 

Throughout the three years, BARR implementation was problematic and provided many 
challenges. Administrators failed to engage fully in the process, the site coordinator and 
teachers did not use technical assistance suggestions to improve performance, and even as 
late as the third year, a negative school climate made it increasingly difficult to create change. 

Because of these challenges, School C provides an excellent opportunity to better 
understand barriers to successful implementation of the BARR model. School C had several 
large school initiatives that were being implemented at the same time that included another 
major grant and a high school construction project. In addition, key administrators in the high 
school changed during the BARR grant and were not as fully involved in implementing and 
supporting the BARR model as were administrators in the other schools. The evaluators noted 
a lack of engagement in the program through informal observations of faculty, students, and 
administrators. Technical assistance providers reported that recommendations to improve the 
program were often not implemented, despite repeated attempts. 
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Chapter VI - Conclusions 

The results of this RCT demonstrated that students in the BARR experimental group 
earned significantly more credits in core classes and achieved significantly more growth from 
fall to spring in NWEA mathematics and reading tests when compared to students in the non- 
BARR group. After the RCT was over, academic performance in the entire 9 th grade 
continued to increase in years 2 and 3. A significant achievement gap between Hispanic and 
non-Hispanic students was evident in year 1, and then closed in years 2 and 3 to reveal no 
significant difference between Hispanic and non-Hispanic students in core course failure rate. 

Another important finding was the association between the quality of implementation 
and student outcomes. The school that did not show changes in student failure rate and 
performance on standardized tests did not have adequate implementation quality ratings, 
while schools that implemented BARR with fidelity showed notable and sustained 
improvements in both academic measures. 

From a scientific perspective, these results are notable given the use of awithin- 
school student-level randomized controlled design, which is relatively rare in educational 
research. This design requires support from school administration, cooperation from 
teachers, and a high level of commitment by all involved. This commitment by the school 
enabled us to test causal outcomes of the BARR model. As part of a review process for an i3 
Validation grant, the results from the RCT year were reviewed by What Works Clearinghouse 
and met their criteria for an evidence-based program without reservations. 

The findings described in this report are consistent with the results of other successful 
9 th grade transition programs. The current study extends the literature by demonstrating a two- 
year growth in standardized mathematics test scores, and by employing a within-school 
randomized controlled design. The BARR model demonstrated that relationship building 
focused on non-cognitive social/emotional supports, combined with rigorous academic 
standards and close attention to student performance produced higher academic achievement 
for students transitioning into high school. 

Research demonstrates that positive school climate, positive relationships between 
students and staff, and among staff, are essential ingredients for turning around low 
performing schools. Research is growing on the effectiveness of student-teacher relationships 
in producing increased attendance and academic performance, and decreased behavior 
problems. With core components derived from research, the Building Assets Reducing Risks 
(BARR) model®targets students at this critical juncture in their academic career—9 th grade. 
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BARR addresses the developmental, academic, and structural challenges of the 9 th grade by 
combining student asset building, teachers’ real-time analysis of student data, and intensive 
teacher collaboration to prevent course failure. BARR develops positive student-teacher 
relationships and integrates student supports into a school's existing model for addressing 
non-academic barriers to learning. Given the importance of a successful transition to high 
school and of forming positive school relationships, these statistically significant findings that 
BARR impacts students’ academic achievement and socio-emotional development, suggest 
deeper exploration of this model is warranted. BARR is the “system” for operationalizing key 
relationship theory and practice. It demonstrates how BARR, as a relationship-focused model, 
leads to continued engagement and increased academic performance for students that come 
to school already connected, and helps those students who have lost that connection to re¬ 
engage. 

As in any study, several limitations need to be considered. For example, BARR 
teachers may have shared new strategies and techniques with other teachers and non- 
BARR students may have benefited from subtle changes in school climate resulting from the 
adoption of this strength based approach. If these events had occurred, although good for 
educational reform, they may have diminished the statistically significant differences 
between BARR and non-BARR students. Further testing of the BARR model in other schools 
will assess the external validity of these findings. 

In terms of future directions, BARR was awarded the U. S. Department of Education 
Investing in Innovation Validation grant: Building Assets Reducing Risks: A Proven Strategy 
to Increase Student Achievement by Improving Teacher Effectiveness in 2013. The grant will 
test the model in 11 RCT schools across the country, and provide materials and training to 
an additional 45 schools. The project is designed so that 51% of participants will be from 
rural areas. The outcome measures will include credits earned in the core subjects, growth 
in NWEA mathematics and reading Scores, and more documentation of student and teacher 
experiences in the BARR Model. 

BARR is a comprehensive model that addresses the challenges that are part of the 9 th 
grade transition year. It is unique in that it is a socio-emotional model that produces significant 
academic results. The model provides the support that students need to be successful in their 
transition to high school and sets them on a positive trajectory toward graduation and beyond. 
Two quotes from superintendents capture the real world significance of the BARR model: 
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"I know that if students are on track in the 9th grade, their chances for graduating on 
time are significantly enhanced." (Rob Metz, Superintendent, St. Louis Park School 
District, MN 

“We’re finding that students have better daily attendance and are staying in school 
instead of dropping out. Equally important, there’s support for teachers from BARR 
educators, who are using in-situation coaching, quarterly site-to-site visits, and 
technology enabled learning opportunities to assist them. We’re creating positive, 
intentional relationships, and using real-time student data in a collaborative problem 
solving process to meet the individual needs of all students. ” (Jim Boothby, 
Superintendent of schools, Regional School Unit 25). 
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Appendix A 


Table la. Attrition for Credits Earned in Core Courses 


Attrition Rate for Confirmatory Contrast Based on the 2011 -12 School Year: 

Outcome = Credits Earned in Core Courses 



BARR Group 

Non-BARR Group 


Assigned sample 

278 

277 


Analytic sample 

272 

276 


Attrition rate 

2.2% 

0.4% 


Overall attrition 



1.3% 

Differential attrition rate 



1.8% 


Table 1b. Attrition for NWEA Reading 


Attrition Rate for Exploratory Contrast Based on the 2011-12 School Year: 

Outcome = NWEA Reading Score 



BARR Group 

Non-BARR Group 


Assigned sample 

278 

277 


Analytic sample 

261 

234 


Attrition rate 

6.1% 

15.5% 


Overall attrition 



1 0.8% 

Differential attrition rate 



9.4% 


Table 1c. Attrition for NWEA Mathematics 


Attrition Rate for Exploratory Contrast Based on the 2011-12 School Year: 

Outcome = NWEA Mathematics Score 



BARR Group 

Non-BARR Group 


Assigned sample 

278 

277 


Analytic sample 

252 

231 


Attrition rate 

9.4% 

16.7% 


Overall attrition 



8.7% 

Differential attrition rate 



7.3% 
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Appendix B 

Table 2. Baseline Measurement - Group Data 



Size of 

Intervention 

Group 

Mean of 

Intervention 

Group 

St. Dev. of 

Intervention 

Group 

Size of 

Comparison 

Group 

Mean of 

Comparison 

Group 

St. Dev. of 

Comparison 

Group 

Total Credits 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

LP - Credits 

170 

216.482 

9.166 

148 

215.041 

1 1.748 

MP-Credits 

100 

233.4 

7.463 

103 

232.65 

6.282 

Total Read 

261 

223.13 

1 1.659 

234 

222.556 

12.831 

LP - Read 

162 

216.809 

8.921 

135 

215.126 

1 1.29 

MP - Read 

99 

233.475 

7.463 

99 

232.687 

6.1 17 

Total Math 

252 

230.286 

14.264 

230 

230.5 

14.42 

LP- Math 

154 

221.545 

9.516 

135 

222.1 1 1 

1 1.522 

MP- Math 

98 

244.02 

8.471 

95 

242.421 

8.522 


LP— Less Proficient; MP — More Proficient; Total Read— NWEA Reading scores; Total Math — NWEA Math 
scores 
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Appendix C 

Mean number of core credits and NWEA scores by Study group, Gender, and 
Hispanic origin 


Table 3a. Mean number of core credits and NWEA scores by Study Group 


Study 

Group 

N 

Core Credits 

Fall 

Math 

Spring Math 

Fall 

Reading 

Spring 

Reading 

BARR 

270 

5.65 

230.063 

236.509 

222.915 

227.429 

Non-BARR 

251 

5.39 

230.886 

232.021 

222.795 

225.491 


Table 3b. Mean number of core credits and NWEA scores by Gender 


Study 

Group 

Gender 

N 

Core 

Credits 

Fall 

Math 

Spring 

Math 

Fall 

Reading 

Spring 

Reading 

BARR 

Females 

146 

5.68 

228.304 

233.971 

222.539 

226.681 

Males 

124 

5.62 

232.75 

240.304 

223.825 

228.842 

Non- 

BARR 

Females 

135 

5.53 

228.22 

229.902 

223.449 

226.819 

Males 

116 

5.24 

233.529 

233.843 

221.495 

223.318 


Table 3c. Mean number of core credits and NWEA scores by Hispanic origin 


Study 

Group 

Ancestry 

N 

Core 

Credits 

Fall 

Math 

Spring 

Math 

Fall 

Reading 

Spring 

Reading 

BARR 

Non- 

Hispanic 

182 

5.78 

231.512 

238.738 

224.534 

229.421 

Hispanic 

88 

5.39 

227.615 

232.551 

220.12 

223.928 

Non- 

BARR 

Non- 

Hispanic 

148 

5.49 

233.478 

234.836 

224.964 

227.791 

Hispanic 

103 

5.26 

226.429 

227.005 

219.032 

221.453 


50 





































Building Assets Reducing Risks 


Appendix D 

Table 4a. Regression predicting core credits earned 

Parameter Estimates 


Parameter 

Beta 

Std. Error 

t 

p-Value 

Study Group 

0.230 

0.082 

2.793 

0.005 

Gender 

0.143 

0.082 

-1.742 

0.082 

Hispanic Origin 

0.161 

0.087 

-1.853 

0.064 

Fall Reading 

0.028 

0.003 

8.369 

0.000 


Table 4b. Regression predicting Spring NWEA scores in Mathematics 

Parameter Estimates 


Parameter 

Beta 

Std. Error 

t 

p-Value 

Study Group 

5.260 

0.710 

7.410 

0.000 

Fall Math 

0.026 

0.025 

36.789 

0.000 

Gender 

0.693 

0.720 

0.963 

0.336 

Hispanic Origin 

-1.871 

0.754 

-2.842 

0.013 


Table 4c. Regression predicting Spring NWEA scores in Reading 

Parameter Estimates 


Parameter 

Beta 

Std. Error 

t 

p-Value 

Study Group 

1.841 

0.638 

2.886 

0.004 

Fall Reading 

0.802 

0.027 

30.209 

0.000 

Gender 

-0.284 

0.637 

-0.446 

0.656 

Hispanic Origin 

-1.763 

0.678 

-2.601 

0.010 
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Appendix E 

Table 5a. Reported Findings - Group Data 



Unadjusted Mean of 
Intervention 

Group 

Unadjusted St. Dev. of 
Intervention 

Group 

Mean of 
Comparison 
Group 

St. Dev. of 
Comparison 
Group 

Total Credits 

5.652 

0.807 

5.394 

1.187 

LP - Credits 

5.518 

0.931 

5.149 

1.367 

MP-Credits 

5.88 

0.456 

5.748 

0.737 

Total Read 

227.674 

1 1.51 

225.218 

12.998 

LP - Read 

222.333 

9.335 

218.415 

1 1.634 

MP - Read 

236.414 

9.164 

234.495 

8.118 

Total Math 

236.782 

16.105 

231.609 

15.168 

LP- Math 

227.617 

10.515 

223.548 

11.787 

MP- Math 

251.184 

12.403 

243.063 

11.684 


LP— Less Proficient; MP — More Proficient; Total Read— NWEA Reading scores; Total Math — NWEA Math 
scores 


Table 5b. Reported Findings - Estimates 



Impact Estimate 

Std. Error of 
Impact 
Estimate 

Two-tailed 

p-value 

Degrees of 
Freedom 

Total Credits 

0.23 

0.082 

0.005 

516 

LP - Credits 

0.288 

0.123 

0.02 

313 

MP-Credits 

0.124 

0.087 

0.157 

198 

Total Read 

1.841 

0.638 

0.004 

490 

LP - Read 

2.486 

0.873 

0.005 

292 

MP - Read 

1.199 

0.91 

0.189 

193 

Total Math 

5.212 

0.696 

0.001 

477 

LP- Math 

4.235 

0.895 

0.001 

284 

MP- Math 

6.24 

1.029 

0.001 

188 


LP— Less Proficient; MP — More Proficient; Total Read— NWEA Reading scores; Total Math — NWEA Math 
scores 
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Appendix F 

Table 6a. OLS Regression predicting core credits earned - Less Proficient group 


Effect 

Coefficient 

Standard 

Error 

Std. 

Coefficient 

Tolerance 

f 

p-Value 

CONSTANT 

-2.141 

1.292 

0.000 


-1.657 

0.098 

1 =Experimental, 
0=Non-BARR 

0.288 

0.123 

0.123 

0.984 

2.345 

0.020 

Gender 

-0.190 

0.124 

-0.081 

0.978 

-1.534 

0.126 

Student is Latino 
(of Hispanic 

Ethnic 

Ancestry) 

-0.208 

0.124 

-0.089 

0.976 

-1.680 

0.094 

Fall NWEA 
Reading 

0.035 

0.006 

0.312 

0.968 

5.877 

0.000 


Table 6b. OLS Regression predicting core credits earned - More Proficient group 


Effect 

Coefficient 

Standard 

Error 

Std. 

Coefficient 

Tolerance 

t 

p-Value 

CONSTANT 

3.895 

1.478 

0.000 


2.635 

0.009 

1 =Experimental, 
0=Non-BARR 

0.124 

0.087 

0.101 

0.986 

1.421 

0.157 

Gender 

-0.026 

0.087 

-0.021 

0.993 

-0.294 

0.769 

Student is Latino 
(of Hispanic 

Ethnic 

Ancestry) 

-0.063 

0.100 

-0.045 

0.987 

-0.633 

0.527 

Fall NWEA 
Reading 

0.008 

0.006 

0.090 

0.990 

1.278 

0.203 


Table 6c. OLS Regression predicting Spring NWEA Reading - Less Proficient group 


Effect 

Coefficient 

Standard 

Error 

Std. 

Coefficient 

Tolerance 

t 

p-Value 

CONSTANT 

67.051 

9.518 

0.000 


7.045 

0.000 

1 =Experimental, 
0=Non-BARR 

2.486 

0.873 

0.1 17 

0.983 

2.846 

0.005 

Gender 

-0.461 

0.882 

-0.022 

0.979 

-0.523 

0.601 

Student is Latino 
(of Hispanic 

Ethnic 

Ancestry) 

-2.091 

0.884 

-0.098 

0.970 

-2.365 

0.019 

Fall NWEA 
Reading 

0.709 

0.044 

0.674 

0.965 

16.261 

0.000 
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Table 6d. OLS Regression predicting Spring NWEA Reading - More Proficient group 


Effect 

Coefficient 

Standard 

Error 

Std. 

Coefficient 

Tolerance 

/ 

p-Value 

CONSTANT 

33.982 

15.550 

0.000 


2.185 

0.030 

1 =Experimental, 
0=Non-BARR 

1.199 

0.910 

0.069 

0.984 

1.317 

0.189 

Gender 

-0.428 

0.906 

-0.025 

0.993 

-0.473 

0.637 

Student is Latino 
(of Hispanic 

Ethnic 

Ancestry) 

-0.868 

1.045 

-0.044 

0.987 

-0.830 

0.407 

Fall NWEA 
Reading 

0.864 

0.067 

0.678 

0.993 

1 2.967 

0.000 


Table 6e. OLS Regression predicting Spring NWEA Mathematics - Less Proficient group 


Effect 

Coefficient 

Standard 

Error 

Std. 

Coefficient 

Tolerance 

t 

p-Value 

CONSTANT 

57.138 

9.588 

0.000 


5 .960 

0.000 

1 =Experimental, 
0=Non-BARR 

4.235 

0.895 

0.187 

0.982 

4.731 

0.000 

Gender 

0.840 

0.916 

0.037 

0.959 

0.917 

0.360 

Student is Latino 
(of Hispanic 

Ethnic 

Ancestry) 

-2.866 

0.905 

-0.126 

0.973 

-3.168 

0.002 

Fall NWEA 
Reading 

0.754 

0.043 

0.700 

0.961 

17.484 

0.000 


Table 6f. OLS Regression predicting Spring NWEA Mathematics - More Proficient group 


Effect 

Coefficient 

Standard 

Error 

Std. 

Coefficient 

Tolerance 

t 

p-Value 

CONSTANT 

-35.535 

1 4.762 

0.000 


-2.407 

0.017 

1 =Experimental, 
0=Non-BARR 

6.240 

1.029 

0.246 

0.982 

6.061 

0.000 

Gender 

0.435 

1.035 

0.017 

0.970 

0.420 

0.675 

Student is Latino 
(of Hispanic 

Ethnic 

Ancestry) 

-0.094 

1.192 

-0.003 

0.994 

-0.078 

0.938 

Fall NWEA 
Reading 

1.149 

0.061 

0.770 

0.969 

18.822 

0.000 
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Appendix G 

Table 7a. Core Course Credits by Gender Over 3 Years 


Year 

Gender 

Number 

of 

students 

Average 
number of 
core credits 
earned 

Number of 
failed core 

courses 

Percent of students 
with at least one 
failure in a core 

course 

Average GPA in 

core courses 

Year 1 Non- 

Girls 

148 

5.40 (90.0%) 

79 

26.4% 

2.82 

BARR 

Boys 

128 

5.1 1 (85.2%) 

103 

38.3% 

2.51 


Girls 

148 

5.68 (94.7%) 

43 

1 9.6% 

3.00 

Year 1 BARR 

Boys 

124 

5.62 (93.7%) 

47 

22.6% 

2.80 


Girls 

261 

5.76 (96.0%) 

63 

1 2.3% 

2.90 

Year 2 

Boys 

256 

5.50 (91.7%) 

129 

22.3% 

2.51 


Girls 

238 

5.76 (96.1%) 

55 

10.5% 

3.05 

Year 3 

Boys 

293 

5.39 (90.0%) 

176 

24.9% 

2.53 


Table 7b. Core Course Credits by Hispanic Origin Over 3 Years 


Year 

Ethnicity 

Number of 
students 

Average 
number of 
core credits 
earned 

Number of 
failed core 

courses 

Percent of 
students 
with at least 
one failure 
in a core 

course 

Average 
GPA in core 

courses 

Year 1 Non- 

Non-Hispanic 

163 

5.37 (89.5%) 

92 

25.2% 

2.83 

BARR 

Hispanic 

113 

5.1 1 (85.2%) 

90 

41.6% 

2.44 


Non-Hispanic 

182 

5.78 (96.3%) 

40 

1 5.9% 

3.00 

Year 1 BARR 

Hispanic 

90 

5.40 (90.0%) 

50 

31.1% 

2.73 


Non-Hispanic 

300 

5.63 (93.8%) 

1 10 

1 6.3% 

2.82 

Year 2 

Hispanic 

217 

5.62 (93.7%) 

82 

1 8.4% 

2.54 


Non-Hispanic 

298 

5.58 (93.0%) 

126 

1 6.4% 

2.77 

Year 3 

Hispanic 

232 

5.55 (92.5%) 

105 

21.1% 

2.75 
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Appendix H 

Fidelity Ratings for Key Components of the BARR Program for Years 1 and 2 



Findings 

2011 - 2012 Year 

Findings 

2012 -2013 Year 

Key 

Components 
on Logic 
Model 

Definition of 
High 

Implementat 

ion 

Definition of 
“implementat 
ion with 
fidelity” at 
program 

level 

% of schools 
at high level 
of 

implementati 

on* 

“Implementat 
ion with 
fidelity” for 
year** 

% of schools 
at high level 
of 

implementati 

on* 

“Implementat 
ion with 
fidelity” for 
year** 

Professional 

Development 

Rating of 7 
out of 9 
based on 4 
objectives 

At least 66% 
of schools 
are high 

33% 

No 

1 00% 

Yes 

Restructuring 

Rating 7 out 
of 9 based 
on 6 

objectives 

At least 66% 
of schools 
are high 

1 00% 

Yes 

1 00% 

Yes 

Parent 

Involvement 

Rating of 7 
out of 9 
based on 4 
objectives 

At least 66% 
of schools 
are high 

33% 

No 

66% 

Yes 

Development 
al Assets 
Curriculum 

Rating 7 out 
of 9 based 
on 3 

objectives 

At least 66% 
of schools 
are high 

66% 

Yes 

100% 

Yes 

Block 

Meeting 

Review 

Rating of 7 
out of 9 
based on 4 
objectives 

At least 66% 
of schools 
are high 

1 00% 

Yes 

100% 

Yes 

Risk Review 

Rating of 7 
out of 9 
based on 5 
objectives 

At least 66% 
of schools 
are high 

1 00% 

Yes 

100% 

Yes 

Whole 

Student 

Emphasis 

Rating of 7 
out of 9 
based on 7 
objectives 

At least 66% 
of schools 
are high 

1 00% 

Yes 

100% 

Yes 

Contextual 

Support 

Rating of 7 
out of 9 
based on 4 
objectives 

At least 66% 
of schools 
are high 

1 00% 

Yes 

100% 

Yes 

Overall 

Fidelity to 
BARR 

Program 

Summary 
score of 56 
based on 8 
key 

components 

At least 66% 
of schools 
are high 

1 00% 

Yes 

100% 

Yes 


*Based on data collection during school year, **Calculated based on % in definition 
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Appendix I 

BARR Teacher Survey 



Strongly 

Disagree 

Disagree 

Neither 

Agree 

Strongly 

Agree 

1. The use of 1-Time has improved my 
relationships with students 

6.4% 

8.5% 

8.5% 

61.7% 

14.9% 

2. The use of data on a weekly basis 
enables me to support students in 
real time 

4.3% 

0% 

4.3% 

42.6% 

48.8% 

3. Training in the developmental asset 
approach has improved my 
perception of student strengths 

4.3% 

8.5% 

1 4.9% 

61.7% 

10.6% 

4. Referral to the risk and asset review 
team of my highest need students 
has enables me to focus more 
effectively on all of my students 

4.3% 

21.9% 

1 0.0% 

36.2% 

27.6% 

5. The use of the BARR block meeting 
process has increased the openness 
and effectiveness of communication 

between teachers and administrators 

6.4% 

6.4% 

6.4% 

21.3% 

59.5% 

6. Educator technical assistance visits 

from other teachers who are 
implementing BARR has increased 
my effectiveness 

4.3% 

6.4% 

44.7% 

34.0% 

10.6% 


1 Table reports all teachers across the BARR project’s responses by rating. Reponses are 
reported as percentages. 
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What participating in a BARR Block Team 
Meeting means to me... 

Strongly 

Disagree 

Disagree 

Neither 

Ag ree 

Strongly 

Agree 

1. 1 like the opportunity to share ideas about 
how to solve student learning problems 

2.1% 

2.1% 

2.1% 

44.7% 

49.0% 

2. 1 feel a little less isolated because 1 am a 

member of a block team 

4.4% 

0% 

6.5% 

30.4% 

58.7% 

3. 1 feel proud of our team’s 
accomplishments for students 

2.1% 

2.1% 

2.1% 

34.0% 

59.7% 

4. 1 get helpful ideas about teacher-student 
relationships for the team 

2.1% 

0% 

6.4% 

34.0% 

57.5% 

5. Participating in a team has increased my 
satisfaction with being a teacher 

4.3% 

2.1% 

8.5% 

36.2% 

48.9% 

6. On our team we have come up with new 
ideas on how to change the school to 
improve student progress 

6.4% 

2.1% 

1 0.6% 

49.0% 

31.9% 

7. 1 have learned new ideas from other 
teachers about how to help students with 
behavior problems 

2.1% 

0% 

8.5% 

55.3% 

34.1% 

8. Being a team member has raised my 
morale at school 

2.1% 

2.1% 

6.4% 

36.2% 

53.2% 

9. 1 have a sense of satisfaction from being 
able to help other teachers solve student 
problems 

4.3% 

0% 

8.5% 

44.7% 

42.5% 

10. In some ways, being a block team 
member helps protect me from “teacher 
burnout” 

8.5% 

6.4% 

1 0.6% 

40.4% 

34.1% 

11. My team gives me a sense of shared 
rather than sole responsibility for students 
with behavior problems 

2.1% 

2.1% 

6.4% 

36.2% 

53.2% 

12. When a student is in real trouble, 1 feel 
like 1 have a team supporting me 

2.1% 

4.3% 

4.3% 

36.2% 

53.2% 

13. Participating on a team has 
strengthened my bonds with other teachers 

2.1% 

0% 

8.5% 

27.7% 

61.7% 
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